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ABSTRACT 


This research was initiated due to a report claiming 
that The U.S. Navy significantly overestimated repair 
prices in a Performance Based Logistics (PBL) reward to a 
contractor. The purpose of this thesis is to develop a 
model for improving the prediction of repair price for U.S. 
Navy inventory items. 

The thesis examines several prediction methodologies, 
including a ratio-estimator prediction method that is a 
modification of the methodology currently in use, as well 
as regression analysis. In contrast to the ratio-estimator 
approach, regression is able to utilize a wide range of 
predictor variables, several of which are evaluated in the 
thesis research. Results of this analysis reveal that a 
regression model with logarithmic transformations yields 
more accurate predictions of repair prices than the current 
methodology. This improvement is seen especially for items 
that have the highest replacement price. 

One feature of the proposed regression-based 
methodology is that predicted repair prices for the most 
expensive items are substantially lower than with the 
current methodology. In the case where that prompted the 
thesis research, the overstatement of benefit from the PBL 
would have been reduced by about 30 million under the 
proposed methodology. 
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EXECUTIVE SUMMARY 


The U.S. Navy has initiated programs to team with 
private industry for logistics support. In particular. 
Performance Based Logistics (PBL) initiatives have been 
undertaken by Naval Inventory Control Point (NAVICP)- 
Philadelphia to achieve its goals for improving support and 
lowering total ownership cost of the Naval inventory items 
under its management. For each PBL initiative, NAVICP 
conducts a business case analysis (BCA) to estimate its 
cost benefits in order to support the decision to award the 
contract. 

This research was initiated as a result of a report by 
the Department of Defense Office of the Inspector General 
(DoD IG) claiming that NAVICP overstated cost saving in the 
F/A-18E/F Integrated Readiness Support Teaming (FIRST) 
Program award to Boeing. The report found that the cost of 
repairs conducted by Boeing were substantially lower than 
what the Navy paid for these repairs, based on its repair- 
price estimates. Estimated repair prices in NAVICP's BCA 
analysis were more than two hundred percent higher than the 
actual repair price reported by Boeing. Most of the 
discrepancy was attributed to repairable items that had the 
highest replacement price ($10,000 and above). 

The thesis research examined alternative methodologies 
for predicting repair prices with the aim of improving the 
prediction accuracy of the methodology currently used by 
NAVICP. The current methodology uses ratio estimators for 
a categorization of repairable items based on their 
replacement price. Within each of six replacement price 
categories, the average of the ratios of repair price to 


xv 



replacement price is calculated, using equal weighting. 
For an item with no historical repair-price data, its 
repair price is predicted by multiplying the appropriate 
ratio by its replacement price. 

A modification of the ratio methodology was 
considered, which weighted the averages proportionally to 
the replacement prices of items within a category. Analysis 
using the modified ratio methodology offered slightly 
improved results for items in the highest replacement price 
category. However, an analysis of both ratio methods found 
that replacement price alone is not a strong predictor of 
repair price. 

An investigation was then made into the use of 
regression models for the prediction of repair prices. 
These models considered variable transformations as well as 
additional predictor variables. The simplest regression 
model used the logarithm of replacement price to predict 
the logarithm of the ratio of repair price to replacement 
price. More extensive regression models included 
categorical predictor variables based on the Federal Stock 
Class (FSC) and Local Routing Code (LRC) of an inventory 
item. Stepwise regression procedures were used to identify 
models that had good predictive power without overfitting. 

Regression models that included combined categorical 
groupings of FSC and LRC yielded the best prediction power. 
However, the level of improvement was small when compared 
to the simple regression model that used only the logarithm 
of replacement price as a predictor. 

The prediction methodologies that were analyzed were 
then evaluated on the repair data cited in DoD 10's report 
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on the FIRST program. The simplest regression model 
resulted in a substantial improvement over the current 
methodology for the inventory items that were considered. 
The total prediction error was reduced from forty-seven 
million dollars (current methodology) to fourteen million 
dollars (regression). 

The thesis research found that a modification of the 
methodology currently used by NAVICP to predict repair 
prices can improve prediction accuracy, particularly for 
items with the highest replacement prices. Because repair 
prices are positively related to replacement prices, 
repairs of these items have the greatest fiscal impact. 
Improved prediction accuracy can be achieved by adopting a 
regression-based methodology, which in addition to 
improving prediction accuracy allows for the inclusion of 
multiple predictor variables. Although FSC was found to 
moderately improve the prediction accuracy, and LRC was 
found not to greatly improve the prediction accuracy of the 
regression models considered, only a limited number of 
potential predictors were examined. It is possible that a 
more extensive investigation would uncover predictors that 
substantially improve the prediction accuracy of these 
models. 


xvii 



THIS PAGE INTENTIONALLY LEFT BLANK 



INTRODUCTION 


I. 


A. INTRODUCTION 


The research described in this thesis is the result of 
an initiative by Naval Inventory Control Point (NAVICP)- 
Philadelphia to replace its current methodology for 
predicting the repair prices of its inventory items. The 
focus of this research is the development of an improved 
methodology so as to provide a more accurate prediction of 
the costs of repairing Naval inventory items. 

B. BACKROUND OF THIS RESEARCH 

In 1999, the Navy adopted an initiative known as the 
Performance Based Logistics (PBL) that transfers 
traditional Department of Defense (DoD) inventory, supply 
chain and technical support functions to commercial 
contractors for a guaranteed level of performance at the 
same or, ideally, reduced cost [1]. Under the PBL program, 
the contractor is responsible for a range of activities 
including warehousing, transportation, repair/overhaul, 
technology insertion, engineering services, guaranteed 
reliability, and warranty management. An example of this 
concept was the award of a contract to The Boeing Company 
(hereafter known as Boeing) on May 2001 for a pilot program 
on F/A-18E/F Integrated Readiness Support Teaming, also 
known as the FIRST Program. 
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For the "FIRST" PBL initiative, NAVICP conducted a 
Business Case Analysis 1 (BCA) based on a five-year period to 
quantify any cost benefits that the Navy might realize due 
to this collaboration. These cost benefits can accrue from 
either cost avoidance or cost savings. The NAVICP BCA, 
using data from FY1995, indicated that an expected saving 
of $55.4 million (later adjusted to $52.4 million) was 
achievable over a five-year period. The analysis also 
anticipated savings of $73.7 million in cost avoidance 
relating to other integrated logistics support elements 
from the FIRST Program [1] . Thus the Navy anticipated a 
saving of $126.1 million contracting of FIRST program. 

However, an audit conducted by the Department of 
Defense Office of the Inspector General (DoD IG) found that 
the business case that the Navy used to justify the award 
of the FIRST contract, overstated the cost of DoD 
performance by $268.9 million. Therefore, instead of the 
Navy's claim of saving $126.1 million over the first five 
years, the analysis showed that the FIRST program actually 
cost the Navy $142.8 million more than the traditional 
support method [1] . Hence NAVICP was tasked to look into 
the accuracy of the business case analysis used extensively 
to develop these numbers. Chapter II provides a more 
detail description of the "FIRST" PBL as well as the BCA 
used in this program. 


1 BCA is a process designed to quantify cost area such as 
procurements, repairable, warehousing, maintenance cost etc used to 
determine Navy's current cost. 
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C. OBJECTIVE AND SCOPE OF RESEARCH 

The research described in the thesis was conducted 
under the sponsorship of the Operations Research group at 
NAVICP-Philadelphia. The aim of the research is to develop 
a predictive model for estimating the repair prices of 
components when the procurement price is known, but no 
actual repair data are yet available. 


Inventory items are classified as either consumable or 
repairable, depending on whether they can be restored to 
ready-for-issue condition in a cost-effective manner when 
they fail. Accurate prediction of the cost of repairing an 
item is an important step in determining whether the item 
should be classified as consumable or repairable. It is 
also important to the effective management of the inventory 
system. 


Repair prices for aviation and weapon system support 
can be difficult to predict. For example, an item could 
be repaired in one of several types of facilities, 
including Navy depots, inter-service depots or commercial 
facilities, thus resulting in large variations in repair 
prices. The ability to obtain accurate predictions and to 
quantify uncertainty in these predictions is important to 
defense organizations, which utilize them to determine the 
resources needed to fully optimize and fund its operations. 
Under-funding could result in a delay or cancellation of 
equipment replacement, whereas over-funding could deny 
resources to other operations. 
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In August 2003, the NAVICP Operations Research group 
was tasked to develop a methodology for improving 
prediction accuracy for repair prices, in light of the 
over-prediction of savings for the FIRST program. Because 
data on repair prices were not always available when 
conducting BCA for the FIRST program, NAVICP established 
the repair prices for newly provisioned items using the 
repair price matrix for estimating the repair prices versus 
historical data for similar items available at the Naval 
depots [1] . The repair matrix is based on a stratification 
of repairable items (mainly aviation and weapon system 
components) by replacement price into six different price 
categories. The ratio of the repair to replacement price 
is estimated within each category. Repair prices for newly 
provisioned items are determined by multiplying the 
replacement price with the appropriate ratio of repair to 
replacement prices. Chapter II discusses the repair matrix 
in detail, which was used as a baseline for comparison in 
the thesis research. 

D. METHODOLOGY 


To facilitate the thesis research, NAVICP provided 
data for 1,420 newly provisioned aviation and weapon system 
inventory items. Information of these inventory items 
include nomenclatures, federal stock class, local routing 
codes, replacement prices and repair prices. Other related 
repair processes and pricing information were subsequently 
obtained from the Material Budgeting and Industrial Support 
Departments at NAVICP. 
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This thesis reviews the appropriateness of various 
statistical analysis methodologies, in an attempt to 
improve the accuracy of repair-price predictions. The 
ratio-estimator model used by NAVICP does not allow for the 
use of other potential explanatory variables that may aid 
in predicting repair prices. Other methods, such as 
regression, can make use of a range of explanatory 
variables. This thesis describes statistical analyses to 
evaluate the benefits of including explanatory variables in 
addition to replacement price. The results of these 
analyses are then compared to predictive accuracy of the 
current methodology. All data analyses are performed using 
the statistical software package S-Plus®. 

E. BACKGROUND OF NAVICP 


The objective of this section is to provide a brief 
understanding of the operations of NAVICP. The mission of 
NAVICP is to provide program and support for the weapon 
systems that will keep its Naval forces mission-ready [7] . 
This mission is carried out by a single command 
organization operating as a tenant activity of Naval 
Support Activities in Mechanicsburg and Philadelphia. 

NAVICP was established in 1995, following the merging 
of Ships Parts Control Center (SPCC) in Mechanicsburg and 
Aviation Supply Office (ASO) in Philadelphia. This merger 
allowed the Navy to bring together all inventory support 
functions under a single command, ensuring a timely 
delivery of quality program and logistics support to keep 
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its Naval forces mission-ready [7] . The merger is also 
part of the Navy's effort to reduce cost and infrastructure 
activities as well as to standardize inventory management 
procedures. 


The primary activity for the Philadelphia site is 
aviation and weapon system support. Among the aircraft 
supported are the F/A-18 and the V-22 as well as various 
engines, common avionics, and support equipment. In 
contrast, their Mechanicsburg counterpart handles support 
for hull, electrical, mechanical, and electronic components 
and repair parts for ships, submarines and weapon systems. 


This section discusses the departmental activities of 
three main offices that are closely related to the accuracy 
of the repair price, namely Material Budgeting Department 
of Comptroller Directorate (P-code 013), Industrial Support 
Department of Operations Directorate (P-code 034) and 
Operations Research Department of Planning and Data Systems 
Directorate (P-code 0412) . Figure 1 gives an overview of 
the NAVICP organization. 
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Figure 1. NAVICP Philadelphia Organization Chart [After 
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The Comptroller Directorate (P-code 01) is responsible 
for controlling the development, preparation and 
justification of all NAVICP budgetary requirements. Its 
Material Budget department administers and controls the 
financial aspects of inventory management, develops and 
justifies initial replenishment as well as repair budgetary 
requirements for Naval ship material support [4] . They are 
the primary users of the repair price matrix used to 
determine the final net price (including inflation, 
overheads, burdening, etc.) charges to the customer. 


The primary functions of the Operations Directorate 
(P-code 03) involve providing supply policies, customer 
liaison, technical support and forecasting/planning of the 
Navy Aviation Operations. The Industrial Support 
Department is responsible for the execution and management 
of the Aviation Depot Level Repairable (AVDLR) repair 
program. The team coordinates component repair work 
executed by the Naval Aviation depots (NADEPs), and the 
Inter-Service component repair work executed by the Army 
Depots and Air Force Logistics Centers [8]. It obtains and 
compiles the repair pricing (from organic depot, 
DMISA/NAPRA and Commercial contractors) required for 
computation of budgetary and contracting requirements. 
Additionally, the team supports NAVAIR Scheduled Depot 
Level Maintenance (SDLM) functions and the Management 
Control Activity (MCA) for Government Furnished Material 
(GFM) contracts, and serves as the focal point for 
information and problem resolution associated with depot 
level repairs. 



The Planning and Data Supply Directorate, P04, 
analyzes, develops and recommends plans, policies and 
procedures to improve practices and activities at NAVICP. 
The Operations Research group is responsible for developing 
the repair price matrix for estimation of repair prices. 
Figure 2 shows the activities of each department relating 
to repair prices. 

Figure 2. Responsibilities of each department relating to repair 

prices. 




P0412 uses the data 
provided by P034 to 
develop the repair 
price matrix 




Customers 


P013 uses the repair 

(Fleet) 

< n 

matrix to determine net 



prices for customers 




F. THESIS ORGANIZATION 


This thesis is divided into five chapters. 


Chapter I gives an introduction to NAVICP and its 
organization responsibilities. It also describes the 
background for this thesis research, the objective and the 
methodologies used to conduct the research. 
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Chapter II discusses in detail NAVICP repair processes 
and its pricing methodologies and Business Case Analysis 
(BCA) used for Performance Based Logistics (PBL). 


Chapter III describes the sources of data used, 
including filtering of unwanted information from the data. 
This chapter also looks into the current ratio methodology 
adopted by NAVICP as well as proposing an alternative ratio 
method that could help improve the prediction power of the 
repair matrix. 


Chapter IV presents the results of the analysis and 
comparisons to the predictive accuracy obtained with the 
current pricing matrix. 

Chapter V summarizes the research findings and 
presents recommendations for future research. 
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II. OVERVIEW OF NAVY REPAIR PROCESS 


A. INTRODUCTION 


This chapter describes in detail the U.S. Navy's 
repair processes (depot-level, commercial or inter-service) 
and its importance to budgeting and contracting. These 
processes were introduced in Chapter I in relation to the 
Performance Based Logistics effort for the "FIRST" program, 
which led to the investigation of the accuracy of its 
current methodology used in its Business Case Analysis for 
predicting repair prices. 

B. OVERVIEW OF NAVICP REPAIR PROCESS 

Repair price refers to the actual unit price that 
NAVICP pays to repair an item. The repair may be performed 
by any one or combination of the following; 

1. Commercial _ facilities _ (definitized _ contract) . 

Commercial repair information can be obtained from the 
Navy's contract status file. This file states the 
level of effort to inspect, repair and modify an item 
for that particular program. 

2. Organic (Navy) depots (NADEPS) . Repair performed at 

one of the three Naval repair facilities located in 
California, Florida and North Carolina. Repair 

pricing consists of actual material cost, labor cost 
and Navy Industrial Fund (NIF) rate 2 . 

2 The NIF rate is established as a cost recovery rate for labor cost, 
overhead costs, transportation, etc. and may vary for different 
Designated Overhaul Points (DOP). 
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3. Depot Maintenance Inter-Service Agreement (DMISA). 


Repairs performed through an inter-service agreement 
with the Army or the Air Force. 


The projected repair price for each repairable item, 
which is stored in data element B055A, is usually updated 
annually and does not actually display the "latest" price. 
NAVSUP directives concerning repair price updates for 
repairable items dictate that it recover any losses over a 
period of approximately twelve-fifteen months. For a given 
item, the repair price in B055A represents a computed 
weighted average repair price for any or all members of the 
family, and from all sources. Thus, each inventory item 
classified as either a head or member of the family, 
depending on the importance of the components in its 
family, will have the same computed weighted average repair 
price. 


Repair prices are determined by computing weighted 
averages of prices, with the weights proportional to the 
number of items repaired at each price category. For 
example, there may be commercial orders and organic 
repairs, for a particular item, from two Navy depots 
(NADEP) that contribute to an average price of $14,190 as 
shown in Table 1. Thus, the fleet could pay $14,190 even 
though the repair price may cost $22,500. 
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Table 1. Computed Weighted Average Pricing Methodology 


NUN 

Nomenclature 

Source 

Qty 

Price 

Total 

011199660 

Amp1ifie r 

Organic 1 

2 

$22,500 

$45,000 


Radio 

Organic 2 

6 

$9, 800 

$58,800 


Commercial 

12 

$15,000 

$180,000 



Total 

20 


$283,800 



Average 



$14,190 


New Repair 

Price 

$14 

i,190 



NUN is the identification number of the inventory item. Source refers 
to where the repair was performed. 


Every year, the Industrial Support Department provides 
the list of organic as well as DMISA repair prices. These 
updated repair prices are then used by the NAVICP 
Operations Research group to generate the repair-price 
matrix to predict repair prices for newly provisioned 
items. The material budgeting department then uses these 
prices to determine the standard and net prices 3 , which 
include cost recovery for transportation, depot washout, 
obsolescence, testing, taxes etc., as illustrated above in 
Figure 2. 


3 Standard Price is the price the customer pays for a new issue for a 
consumable or Aviation Depot Level Repairable (AVDLR) item. Net Price 
is the price the customer pays for an AVDLR with carcass turn-in. These 
prices are determined by multiplying a cost recovery rate to the repair 
prices. 
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C. PERFORMANCE BASED LOGISTICS FOR F/A-18E/F AIRCRAFT 

Under the PBL program, NAVICP awards a contract to a 
single supplier. This supplier then provides the material 
directly to the fleet in time to meet the customer's 
requirement. This is achieved without the intervention of, 
or need for, government inventory managers or intervening 
storage and material handling systems while providing 
increased product reliability and reducing total cost to 
the Fleet Customer and the Navy [6]. One of first 
contracts issued under the PBL program on May 2001 was for 
the F/A-18E/F Integrated Readiness Support Team program. 
NAVICP awarded Boeing a five-year contract with an award- 
fee provision based on performance requirements. The 
contract covered procurement of initial and replenishment 
spares for 519 repairable parts and 5856 consumable parts 
as well as repair of the repairable parts [1]. 

For each PBL initiative, NAVICP and NAVAIR conduct a 
Business Case Analysis (BCA) to justify the amount of cost 
savings or cost avoidance 4 benefits through the contract. 
The BCA process involves determining the Navy's current 
cost of doing business. For the FIRST Program, the 
"without FIRST" cost is compared with the cost to the Navy 
under a PBL arrangement. The "with FIRST" cost includes 
both the PBL supplier's costs as well as residual costs 
that the Navy will retain even under a PBL arrangement. 
Some cost areas considered in the BCA are depot repair, 
wholesale and retail spare parts procurements, warehousing, 

4 Cost avoidance is calculated based on overheads, operations and 
labor saving by NAVICP due to logistic support. 
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transportation, fleet maintenance labor, fleet consumables 


sustaining engineering, NAVICP operating costs, PBL 
administrative costs and other miscellaneous costs [5]. 


A BCA conducted by NAVICP indicated that an expected 
savings of $55.4 million (later adjusted to $52.4 million) 
was likely over a 5-year period from the FIRST program. 
The analysis also found that savings of $73.7 million in 
cost avoidance relating to other integrated logistics 
support elements from the FIRST program was achievable. 
Table 2 summarizes the Navy's reported five-year $126.1 
million cost avoidance relating to the FIRST contract with 
Boeing [1]. 


Table 2. FIRST Program Savings (1999) - Without FIRST and With 

FIRST (in millions of dollars) [1]. 


Description 

Without 

FIRST 

With 

FIRST 

Cost Increase/ 
(Cost Avoidance) 

NAVICP BCA 




Material costs 

$779.00 

$771.50 

$(7.50) 

Operations cost 

$108.10 

$63.20 

$(44.90) 

Subtotal NWCF cost 

$887.10 

$834.70 

$(52.40) 

NAVAIR BCA 




Non-supply support 
elements 

$1,531.20 

$1,457.50 

$(73.70) 

Total 

$2,418.30 

$2,292.20 

$126.10 


Non-supply support elements refer to cost avoidance due to engineering, 
integrated logistic support, information systems and supporting 
equipment. Cost avoidance is calculated based on overheads, operations 
and labor saving by NAVICP due to logistic support. 
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However, DoD IG found that these claimed savings were 
overstated, and that the FIRST program actually resulted in 
a cost increase of $142.8 million. Table 3 summarizes the 
adjustments made to the BCA [1] . The huge difference in 
the estimation of cost savings led to an investigation. It 
appears that NAVICP used an outdated repair price matrix to 
perform the business case analysis for determining repair 
price of an item with little or no historical cost data 
(i.e. newly provisioned items) [1]. 


Table 3. DoD IG Corrected FIRST Program Savings [1]. 


Description 

IG 

Corrected 

IG Corrected 

Cost 

Increase/ 

Without FIRST 

With FIRST 

(Cost 

Avoidance) 

NAVICP BCA 






Material costs 

$ 

573.80 

$ 783.10 

$ 

209.30 

Operations cost 

$ 

119.50 

$ 63.20 

$ 

(56.30) 

Subtotal NWCF cost 

$ 

693.30 

$ 846.30 

$ 

153.00 

NAVAIR BCA 






Non-supply support 
elements 

$ 

1,531.20 

$ 1,521.00 

$ 

(10.20) 

Total 

$ 

2,224.50 

$ 2,367.20 

$ 

142.80 


Non-supply support elements refer to cost avoidance due to engineering, 
integrated logistic support, information systems and supporting 
equipment. Cost avoidance is calculated based on overheads, operations 
and labor saving by NAVICP due to logistic support. 
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D. THE NAVICP METHODOLOGY FOR PREDICTION OF REPAIR PRICES 

This section describes how NAVICP uses actual repair 
prices to generate the repair price matrix, and the 
importance of these costs to the accuracy of this matrix. 


NAVICP developed the repair price matrix in 1986, 
based on a study of actual repair prices of items across 
all weapon systems and aviation items that had procurements 
in the previous two years. The repair matrix groups 


inventory 

items based on 

replacement 

price into 

six 

categories. 

Within each 

category. 

repair price 

is 

predicted 

by applying a 

common multiplier to 

the 


replacement price. The multiplier is calculated by taking 
the average of the ratios of repair to replacement price. 
For a newly provisioned item (that has not generated a 
repair history), its repair price is predicted by 
multiplying its replacement price by the appropriate 
multiplier. For example, the multiplier for items with 
replacement prices less than $999 (Category 1) is 48%, and 
an item with a replacement price of $800 has a predicted 
repair price of $384.00 (48% of $800) . This repair price 

matrix used by NAVICP in justifying cost saving from the 
FIRST program is shown in Table 4. 
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Table 4. Repair Price Matrix used for FIRST Program in 1995 

[After, 1]. 


Replacement Price of 
Inventory Item (dollars) 

Predicted Repair Price, as 
Percentage of Replacement Price 

1 - 999 

48 

1,000 - 2,999 

32 

3,000 - 9,999 

30 

10,000 - 24,999 

24 

25,000 - 49,999 

20 

50,000 + 

15 


The percentage in the second column is the average of the ratios of 
repair to replacement price across all items in the corresponding 
category. 
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III. OVERVIEW OF CURRENT METHODOLOGY 

A. INTRODUCTION 

This chapter describes the source of data used in the 
analysis. It also describes the current prediction 
methodology adopted by NAVICP, and it proposes an 
alternative method that could help improve prediction 
accuracy. 

B. DATA USED IN THIS RESEARCH 

The data used for the cost estimation are obtained 
from the Master Data File from NAVICP's Oracle server. 
This data file is widely used in NAVICP for tracking 
inventories, quarterly demands, replacement and repair 
prices etc. The data also include other variables that may 
be useful in predicting repair prices. An excerpt of the 
data used for the analysis in this thesis is shown in Table 
5 . 
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Table 5. An Excerpt of the Data Used in the Thesis Research. 


COG 

FSC 

NUN 

NOMENCLATURE 

REPL_ 

PRICE 

RPR_ 

PRC 

FRC 

FGC 

LRC 

7R 

5930 

000083636 

SWITCH ASSEMBLY 

4242 

1081 

H 

W6AB 

QAC 

7R 

5985 

000014545 

SWITCH 

10412 

1296 

H 


DMF 

7R 

6110 

000016632 

REGULR,VOLTAGE 

19021 

648 

H 


DTQ 

7R 

1620 

000049840 

PISTON ASSEMBLY 

31093 

9179 

H 


MXD 

1R 

2840 

998149318 

RING, COMPRESSOR 

6805 

0 

H 


SVA 

7R 

1560 

008666688 

TANK,FUEL, 
AIRCRAFT 

20210 

3998 

H 


LHF 

OR 

6130 

014663481 

POWER SUPPLY 

695 

144 

H 


T16 

7R 

5998 

012019353 

CIR.CARD ASSY 

5352 

739 

H 


Q3C 


COG is the Cognizance Codes which is used to identify if item is 
repairable or consumable. FSC is the Federal Stock Class assigned to 
each item to classify them into different categories. NUN is the 
National Item Identification number used to identify an approved 
inventory item. FRC is the Family Relationship Code which identifies 
if an item is the head or member of the family. FGC is the Family 
Group Code used to identify an item within a family. LRC is the Local 
Routing Code used to determine the internal organizational component 
(by Platform) to which item inputs are to be routed for action. 


The first three fields (COG, FSC and NUN) give the 
identity of the supply item. COG is the Cognizance Code, 
which is used to identify an item as repairable or 
consumable. Items with COG code 'OR' are funded and 
managed through special appropriations. Code '1R' refers to 
items that are non-repairable, also known as consumables. 
Code '7R' refers to items that are repairable. 


Federal Stock Class (FSC) is a four digit code 
assigned to each item to classify them into different 

categories, such as aircraft fuel pumps, compressors or 

circuit boards. Appendix A shows a full listing of FSC 

codes used in this analysis. The national item 
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identification number (NIIN) is an eight-digit code used to 


uniquely identify an inventory item. The family 

relationship code (FRC) identifies an item as either the 
head or member of its family. Each family of items is 


identified 

by a 

family group code 

(FGC) 

that 

is blank if 

the family 

consists 

of a 

single 

item 

(also 

known as a 

bachelor item). 

The 

local 

routing 

code 

(LRC) 

is a three- 


letter code used to determine the internal organizational 
component to which item inputs are to be routed for action. 
The first letter of the code identifies the platform group 
that manages the items. Appendix A shows a full listing of 
LRC codes used in this analysis. 


The data set includes a total of 1,420 newly 
provisioned inventory items available from January to June 
2003. Not all the information in the data set provided by 
NAVICP is appropriate for this analysis. Table 6 shows a 
list of items to be excluded from the analysis. 


Table 6. Filtering of data set to be used for this analysis 


Total number of Inventory Items 

1420 

Less 


1. Items with COG = 'OR' or '1R' 

21 

2. Items with lack of data or Repair Price < 0 

6 

3. Items that are not head of the family 

50 

Total number of Items deemed useful for this analysis 

1343 
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Items with COG code 'OR' are funded and managed 
through special appropriations whilst items with Code 'lR' 
are categorized as non-repairable, also known as 
consumables. Hence, predicting the repair prices for these 
twenty one items would not be of fair representation or 
accurate. Only items with Code 7R, items that are 
repairable, are considered in this analysis. 


There are four inventory items that have no 
information other than the inventory identification number. 
Also, there are two items with no repair price information 
and, therefore, these data are removed from the data set. 


NAVICP groups repair price data by families of items. 
Each item within a family has the same estimated repair 
price. To avoid using redundant information, only the data 
reported for the head of the family are used. Fifty items 
corresponding to non-head members of families are removed 
from the analysis. 
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c. 


CURRENT PREDICTION METHODOLOGY 


The current methodology adopted by NAVICP is to 
determine the average of the ratios of repair to 
replacement price across all inventory items within a 
specified category. There are a total of six different 
categories based on replacement price. The following shows 
the calculation used to determine the average cost ratio 
for category j : 


A=2X 


i =1 


A 

x„ 


where Y.. and X H are the repair and replacement prices, 
respectively, for item i in category j . In this notation a 
weight denoted is also used to calculate the average. 

NAVICP uses equal weights, W ij =\/n j , to calculate the 
average ratio, where n j is the number of items in category 
j . The predicted repair price for an item having no 
historical repair price data, with replacement price X 
belonging to category j , is then given by Y = Pj X ■ 


Based on the data provided by NAVICP (total of 1,420 
inventory items), the current repair matrix is shown in 
Table 7. This table also presents a summary of the 
variability and correlation between repair and replacement 
prices for these items. From Table 7, it is observed that 
there is moderate correlation between replacement and 
repair prices for majority of the price categories, which 
indicates that a strong linear relationship between the two 
variables is unlikely to exist. This implies that using 
replacement price as the sole variable to predict the 
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repair price will leave unexplained much of the variability 
in repair prices. 


Table 7. Current Repair Price Matrix 


Price Category 

Number 

of Items 

Correlation 
between repair and 
replacement price 

Predicted Repair 
Cost, as 
Percentage of 
Replacement Price 

UP TO $999 

33 

0.561 

60.91 

$1,000 TO $2,999 

188 

0.269 

42.52 

$3, 000 TO $9,999 

457 

0.238 

34.01 

$10,000 TO $24,999 

354 

0.282 

25.14 

$25,000 TO $49,999 

155 

0.281 

18.44 

$50,000 AND ABOVE 

156 

0.283 

16.00 

Overall 

1343 

0.447 

- 


The percentage in the last column is the average of the ratios of 
repair to replacement price across all items in the corresponding 
category. 



D. MODIFIED RATIO METHODOLOGY 

An alternative ratio method is considered where the 
weights W- are chosen to be proportional to . These 
ratio estimators take the following form: 


where Y- and Xj are the average repair and replacement 
prices, respectively, for all items in category j . Items 
with higher replacement prices in a category are assigned 
more weight using the modified ratio method than with the 
method currently used by NAVICP. Results of applying the 
modified ratio method to the 1,343 items used in the 
analysis are shown in Table 8. 


Table 8. Repair Price Matrix (Modified Ratio Method) 


Price Category 

Number 

of Items 

Predicted Repair 
Price, as Percentage 
of Replacement Price 

UP TO $999 

33 

59.36 

$1,000 TO $2,999 

188 

41.68 

$3, 000 TO $9,999 

457 

32.56 

$10,000 TO $24,999 

354 

24.90 

$25,000 TO $49,999 

155 

18.64 

$50,000 AND ABOVE 

156 

12.40 

Overall 

1343 

- 


The percentage in the last column is the average of the ratios of 
repair to replacement price across all items in the corresponding 
category. 
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E. 


A COMPARISON OF RATIO PREDICTION METHODOLOGIES 


A box plot provides an excellent way to visualize many 
aspects of a distribution in a data set, particularly 
determining the median and variation changes between 
different categories. Figure 3 and Figure 4 show the 
median, variability and outliers of the relative prediction 
error (between repair and predicted repair prices) using 
the current and modified ratio methodology respectively. 
Relative prediction error is defined as follows: 


M) 


It should be noted that data points with relative 
prediction error greater than 0.7 are eliminated to enhance 
graphical display. 


It is observed that the spread increases with each 
price categories and large error can be seen in the region 
of the higher priced categories for both plots. For both 
methodologies, the median and variations are reasonably 
similar for price categories below $50,000. Slight 
improvement in the prediction error can be seen in the last 
categories for inventory items with replacement prices of 
$50,000 and above. 
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A total of 37 out of 1343 
prediction error greater t 
enhance graphical display. 


plots of Relative Pre< 
for the Modif: 



Up to $999 $1,000-$2,999 $3,000-! 


3 

t 

y- 


A total of 33 out of 134! 
prediction error greater 
enhance graphical displa; 







In Table 9, it is rather evident that both methods 
show little success when predicting repair prices for 
inventory items with replacement prices greater than 
$10, 000. Again, the largest error can be seen in the last 
category, whereby 75 percent of the observations have 
approximately 245 percent prediction errors for the current 
ratio method. The modified ratio method shows an 
improvement, but still, the relative prediction errors are 
greater than or equal to 160 percent for almost 25 percent 
of items in this category. 


Both ratio methods indicate that replacement price 
alone may be a poor predictor of the repair price, and that 
the relationship between the two analyzed variables may be 
non-linear. Chapter IV investigates the use of a 
logarithmic transformation in the regression method, as 
well as including categorical variables, that may help to 
improve the prediction of repair prices. 
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Table 9. Quantiles of Absolute Relative Prediction Error (Percentages) of current and 

modified ratio method. 


Category 

Sample 

Size 

Current Ratio Method 

Modified Ratio Method 

25th 

Percentile 

Median 

75th 

Percentile 

Mean 

25th 

Percentile 

Median 

75th 

Percentile 

Mean 

up to $999 

33 

16.47 

28.96 

38.66 

32.59 

16.48 

25.95 

46.35 

36.31 

$1,000 - $2,999 

188 

21.37 

36.94 

75.59 

71.28 

20.91 

37.07 

73.92 

70.39 

$3,000 - $9,999 

457 

22.70 

47.12 

106.44 

96.90 

22.08 

45.62 

99.72 

93.29 

$10,000 - $24,999 

354 

23.33 

52.63 

140.33 

131.67 

23.42 

52.52 

139.34 

131.04 

$25,999 - $49,999 

155 

30.74 

61.38 

146.42 

170.38 

30.22 

62.14 

141.60 

166.56 

$50,000 and above 

156 

40.26 

78.18 

245.32 

320.73 

37.21 

65.14 

160.36 

234.88 
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IV. METHODOLOGY 


A. INTRODUCTION 


In this chapter attention is focused on the use of 
regression models to predict the repair price of an 
inventory item from its replacement price and possibly 
other explanatory variables. This chapter will look at 
regression analysis with logarithmic terms as well as 
categorical variables such as Federal Stock Class, Family 
Relationship Code and Local Routing Code that may help to 
improve prediction accuracy. 

B. REGRESSION ANALYSIS 


Regression analysis is used to develop a mathematical 
model that describes the relationship of repair prices of 
inventory items to other variables. Regression models 
considered in the thesis research are linear, possibly 
after taking nonlinear transformations of the response 
and/or explanatory variables. Scatter plots are examined 
in order to better understand the relationship between 
variables. Figure 5 shows the scatter plot of repair 
prices versus replacement prices on a logarithmic scale. 
The relationship between the two variables can be 
reasonably described as linear on a logarithmic scale. The 
decision was therefore made to examine regression models 
with logarithm transformations applied to these two 
variables. 
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Figure 5. Scatter plot of repair prices versus replacement 

prices for 1,343 repair items in the Naval inventory. 



102 103 10 4 10 5 10 6 


Replacement Price 

The simple logarithmic transformed regression 
(Model I) is expressed as follows: 


model 


log 

V 


Y, 


X 


t y 


A) + A i°g ( X i )+ £ i t 


where X i and Y t are the replacement and repair prices, 
respectively, for inventory item i . The intercept and slope 
coefficients are J3 0 and , and £ j is the error term. It is 
assumed that the error terms £ t are independent, normally 
distributed random variables with mean zero and a common 
(but unknown) variance <J~ . Least-squares regression was 
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used to estimate the coefficients of the regression model, 
using the statistical software package S-Plus @ on data 
consisting of replacement and repair prices for 1,343 Naval 
inventory items. The least squares estimates are shown in 
the following table: 

Table 10. Least-squares Regression Coefficient for the Simple 
Regression Model, Based on n = 1,343 Items. 


Coefficient 

Estimate 

Standard Error 

t-ratio 

Intercept ( jB 0 ) 

1.5858 

0.1571 

10.0687 

Slope (/?, ) 

-0.3403 

0.0168 

-20.2806 


Values of the t-ratio that exceed 1.96 in absolute value are 
statistically significant at the a = .05 test level. 


From Table 10, it is seen that the logarithm of 
replacement price is a statistically significant predictor 
of the logarithm of the ratio of repair price to 
replacement price. To predict the repair price the 
following nonlinear transformation is used: 


t = exp (/? 0 + P x log X t ). 


C. REGRESSION ANALYSIS WITH CATEGORICAL VARIABLES 

One of the advantages of regression models compared to 
ratio-estimator models is the ability of the former to 
incorporate predictor variables of diverse types. NAVICP 
collects data on a wide range of attributes for the 
repairable items that it manages, much of which is 


categorical. In the data set provided by NAVICP, as shown 
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in Table 5, are attributes such as Federal Stock Class 
(FSC) and Local Routing Code (LRC) that groups these 
inventory items into their respective functions and/or 
platform types. In this section regression models that 
incorporate these categorical attributes as predictors is 
examined. 


In the data provided by NAVICP, restricted to 1,343 
inventory items that were used in analyses, a total of 101 
different FSC levels and 18 different LRC levels were 
represented. For model-building purposes only those levels 
with twenty or more inventory items were considered as 
potential predictor variables. These levels are shown 
below in Table 11. 
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Table 11. FSC and LRC levels with twenty or more inventory 

items. 


FSC 

Description 

Number of 
inventory 
items 

LRC 

Description 

Number of 
inventory 
items 

5998 

Circuit Card 

Assemblies 

240 

A$$ 

F/A18 A/D 

205 

1560 

Airframe 

Structural 

Components 

126 

c$$ 

AV-8B 

69 

2840 

Aircraft Engines 
and Turbines 
Components 

102 

D$ $ 

EA-6B 

64 

1615 

Helo Rotor Blades 

Mechanisms 

74 

E $ $ 

F—14 

77 

1650 

Aircraft Hydraulic 
Components 

72 

H$ $ 

S-3 

48 

1620 

Aircraft Landing 
Gear Components 

60 

L$ $ 

P-3 

74 

1680 

Miscellaneous 

Aircraft 

Components 

58 

M$ $ 

E—2 

69 

5895 

Miscellaneous 

Communication 

Equipment 

41 

Q$$ 

Common Systems 

78 

6625 

Electronic Testing 
Instruments 

39 

s$$ 

Aircraft 

Engines 

130 

6130 

Non-rotating 

Electrical 

Converters 

35 

T$ $ 

Aviation 

Support 

Systems 

83 

5985 

Antennas Related 
Equipment 

26 

u$$ 

Aviation 

Support 

Systems 

101 

4920 

Aircraft 

Maintenance 

Equipment 

22 

v$$ 

H-l/H-4 6 

98 

6150 

Miscellaneous 
Electrical Power 
Equipment 

22 

x$$ 

H-3/H-56 

122 

2915 

Aircraft Fuel 
Systems Components 

21 

Y$$ 

H-60 

97 

5841 

Airborne Radar 
Equipment 

20 



LRC is represented by the first character of the code. For example, 
A$$ refers to all LRC codes that begin with the letter "A". 
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F-tests for significance of regression were performed 
to determine if FSC and LRC contributed significantly to 
the prediction of log (Y i /X j ). Models II and III include FSC 
and LRC, respectively, as categorical variables while Model 
IV includes a combination of both FSC and LRC variables. 
These models were fit two different ways: 

Intercept-only models . These models incorporate FSC 
and/or LRC as indicator variables, and the logarithm of 
replacement price as a numeric predictor. Intercept-only 
models regression allow for the possibility that only the 
intercepts of the regressions vary with the categorical 
predictors. 


Intercept-and-slope models . These models incorporate 
the same predictor variables as the intercept-only models, 
and the products of the indicator variables with the 
logarithm of replacement price as predictors. Intercept- 
and-slope models allow both the intercepts and slopes of 
the regressions to vary with the categorical predictors. 


Table 12 tabulates the result of the F-test for 
intercept for Models II, III, and IV fit as intercept-only 
models. For all three models, the null hypothesis is 
rejected, thus it can be concluded that using different 
intercept terms for FGC and LRC categories helps to predict 
repair prices. 
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Similarly the models with slope terms are tested under 
the null hypothesis that the slope terms do not help to 
improve prediction power over their intercept-only 
counterparts. As shown in Table 13, the inclusion of slope 
terms leads to a statistically significant improvement in 
prediction power for Models III and IV, but not for Model 
II. 


The next step is to perform stepwise regression to 
select a subset of variables that are significant as 
predictors. A commonly used form of stepwise regression is 
forward inclusion. Using statistical software package 
S-Plus®’, the default method of Efroymson's forward selection 
procedure is employed. This selection procedure starts 
with an empty subset and at each step adds the independent 
variable that gives the largest reduction of the residual 
sum of squares. As each variable is added to this subset, 
partial correlations are considered to see if any of the 
variables in the subset should be dropped [11] . Table 14, 
15 and 16 summarize the regression models by reporting 
their coefficients, standard errors and t-ratios. 
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Table 12. F-test for Categorical Predictor Models, Intercepts Only 


Model 


(FGC) 


(LRC) 


(FGC and LRC) 


Unexplained variation for the full model, SSEo 
Degrees of Freedom, P 0 

Unexplained variation for Model I, SSEi 


818.980 


:92.553 


852.124 


! 92.553 


795.284 


192.553 


Degrees of Freedom, P] 


Test statistic value, F-test = 


SSE t - SSE () 
Po ~ Pi 


7.941 


4.497 


5.533 


l n ~ Po ) 

Test statistic critical value with significance 
level of 0.05, F n < n = 1,343 

_ OlA)-Pi,»-Po ' _ 

Conclusion (a = .05) 


1. 639 


1.699 


1.476 


Conclusion (a = .05) H 0 Rejected H 0 Rejected H 0 Rejected 

The null hypothesis is that the additional predictors in Models II, III, or IV do not 
improve the prediction power of Model I. 


Table 13. F-test for Categorical Predictor Models, Intercepts and Slopes 


Model 

Unexplained variation for the full model, SSEo 
Degrees of Freedom, P 0 

Unexplained variation for the intercept-only 
models, SSEi 

Degrees of Freedom, Pi 

( SSE.-SSE A 


Test statistic value, F-test = 


Po~Pi 


V n ~Po J 

Test statistic critical value with significance 

-i i f n n; i i/ii 1.674 1.699 1.476 

level of 0.05, n = 1,343 

_ ■05,Po-Pi'”-Po ' _ 

Conclusion H 0 Accepted H 0 Rejected H 0 Rejected 

The null hypothesis is that the slope terms in Models II, III, or IV do not improve the 
prediction power over the intercept-only counterparts. 


II 

III 

IV 

(FGC) 

(LRC) 

(FGC and LRC) 

805.843 

825.412 

766.041 

32 

30 

60 

818.980 

852.124 

795.284 

17 

16 

31 

1.425 

3.035 

1.689 

1.674 

1.699 

1.476 

Accepted 

H 0 Rejected 

H 0 Rejected 



Table 14. Stepwise Regression of Best Possible Selection for 

Predictor using only intercept terms for Model II (FSC 

grouping). 


Coefficient 

Estimate 

Standard Error 

t-ratio 

Intercept 

1.8154 

0.1567 

11.5842 

Log(Xi) 

-0.3700 

0.0166 

-22.2745 

FSC Code 5998 

-0.2741 

0.0591 

-4.6355 

FSC Code 1560 

0.3323 

0.0762 

4.3616 

FSC Code 1615 

0.5104 

0.0960 

5.3178 

FSC Code 1620 

0.3800 

0.1055 

3.6022 

FSC Code 1680 

0.3039 

0.1071 

2.8378 

FSC Code 6625 

0.3128 

0.1291 

2.4224 

FSC Code 5841 

-0.5524 

0.1783 

-3.0978 

R 2 

0.2954 


Values of the t-ratio that exceed 1.96 in absolute value are 
statistically significant at the a = .05 test level and are used as 

criteria for adding variables to the subset when using Efroymson's 
method. 

Table 15. Stepwise Regression of Best Possible Selection for 
Predictor Using both intercept and slope terms for 
Model III (LRC grouping). 


Coefficient 

Estimate 

Standard Error 

t-ratio 

Intercept 

1.6890 

0.1685 

10.0215 

Log(Xi) 

0.1981 

0.0656 

3.0226 

LRC Code E 

-0.2754 

0.0956 

-2.8822 

LRC Code U 

0.2025 

0.0836 

2.4215 

LRC Code X 

-1.0895 

0.5503 

-1.9800 

LRC Code Y 

-1.3794 

0.6097 

-2.2622 

LRC Code A x Log(Xi) 

-0.3574 

0.0180 

-19.8133 

LRC Code Y x Log(Xi) 

0.1411 

0.0589 

2.3956 

R 2 

0.2699 


Values of the t-ratio that exceed 1.96 in absolute value are 
statistically significant at the a = .05 test level and are used as 

criteria for adding variables to the subset when using Efroymson's 
method. 
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Table 16. Stepwise Regression of Best Possible Selection for 
Predictor using both intercept and slope terms for 
Model IV (Includes both FSC and LRC groupings). 


Coefficient 

Estimate 

Standard Error 

t-ratio 

Intercept 

1.6598 

0.1567 

10.5913 

Log(Xi) 

0.0371 

0.0092 

4.0498 

FSC Code 1560 

0.3307 

0.0759 

4.3556 

LRC Code E 

-0.2728 

0.0928 

-2.9407 

LRC Code U 

0.2572 

0.0820 

3.1343 

FSC Code 5998 x Log(Xi) 

-0.3562 

0.0167 

-21.3800 

FSC Code 1560 x Log(Xi) 

-0.0317 

0.0065 

-4.8461 

FSC Code 1650 x Log(Xi) 

0.0430 

0.0101 

4.2616 

FSC Code 1680 x Log(X ± ) 

0.0461 

0.0110 

4.2134 

FSC Code 5895 x Log(Xi) 

0.0335 

0.0115 

2.9208 

LRC Code A x Log(Xi) 

-0.0528 

0.0174 

-3.0336 

LRC Code U x Log(X ± ) 

0.0214 

0.0108 

1.9815 

R 2 

0.3146 


Values of the t-ratio that exceed 1.96 in absolute value are 
statistically significant at the a = .05 test level and are used as 

criteria for adding variables to the subset when using Efroymson's 
method. 


To predict the repair price the following 
transformation is used: 




For Model II(FSC Code with 
Y t = X t exp (A + A log x i + A2A998,/ + A A 560, 


intercept terms only): 

+ AAl615 ,/ + A A 620,1 + A) A 680,/ + A 7 A)625,/ + AA 584 I,/ ) 


where, for example, 

J1, item i has FSC code 5998 
5998,7 [o, otherwise 

and other FSC codes are represented similarly. 

• For Model III (LRC Code with both intercept and slope 
terms): 


Y; = X,- 


exp ( Ao + A log A + Pi l e,i + + + fo$L Y j + As L A i log X t + ^Ly t log X t j 
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where, for example 


|l, item i has LRC code beginning with "A" 
Aj [0, otherwise 

and other LRC codes are represented similarly. 


• Model IV (FSC and LRC Code with both intercept and slope 
terms) 


= ^ exp (p 0 + A log A,. + A /7 ] 560 , + h L E,i 
4 650,i log A,. + faF ia0J log A,. + M 895 , 


+ 44\; + ft 5 F 599Si log A. + ft 6 F l560i log A ; 

logX, + logZ, + A,i„, log- X,) 


D. RESULTS OF ANALYSIS 

Figure 6 shows box plots of the relative prediction 
errors for each of the four regression models that were 
considered. Again, Model I use only log(A) (the logarithm 
of replacement price) as a predictor variable for log(T / A) 
(the logarithm of the ratio of repair price to replacement 
price) . Model II includes FSC categories into the model, 
Model III includes LRC categories into the model, and Model 
IV includes both FSC and LRC categories into the model. 
Relative prediction error is given by the following: 


R, = 


Y-Y ; 


Y 


where 


Y, = A ( exp(Z ; ), 

and 4 is the fitted value of log(l^/A ; .). 
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The box plots indicate that the spread in relative 
prediction errors increases moderately with the replacement 
price; i.e., more expensive items have larger prediction 
errors not only in an absolute sense, but also as a 
percentage of the replacement price. However, relative 
errors in the highest price category are reduced when 
compared to the modified ratio method as described in 
Chapter III. Model I, II and IV displayed rather similar 
results in terms of relative prediction error in the 
highest price category. Regression with the inclusion of 
categorical variables shows slight improvement in relative 
prediction error. 


Table 17 also indicates that for 75 percent 
observations, the relative prediction error ranges 
percent to 96 percent. 


of the 
from 89 
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Table 17. Quantiles of Absolute Relative Prediction Error (Percentages) for Four Regression 

Models Considered. 


Category 

Sample 

Size 

Model I 

Model II 

2 5 th 

Percentile 

Median 

7 5 th 

Percentile 

Mean 

2 5 th 

Percentile 

Median 

7 5 th 

Percentile 

Mean 

up to $999 

33 

18.05 

28.39 

44.80 

39.70 

21.76 

28.98 

56.54 

49.44 

$1,000 - 
$2,999 

188 

19.23 

37.75 

60.45 

59.23 

18.93 

37.41 

59.03 

57.18 

$3,000 - 
$ 9, 999 

457 

24.61 

45.38 

69.28 

71.19 

22.37 

43.92 

66.20 

67.55 

$10,000 - 
$24,999 

354 

26.68 

54.05 

79.23 

94.11 

26.21 

49.93 

73.41 

86.23 

$25,999 - 
$49,999 

155 

31.45 

53.22 

84.99 

121.40 

26.81 

52.43 

85.52 

114.11 

$50,000 
and above 

156 

31.20 

58.41 

95.84 

148.67 

28.53 

60.46 

95.28 

137.95 

Category 

Sample 

Size 

Model III 

Model IV 

2 5 th 

Percentile 

Median 

7 5 th 

Percentile 

Mean 

2 5 th 

Percentile 

Median 

7 5 th 

Percentile 

Mean 

up to $999 

33 

19.16 

28.83 

49.98 

40.65 

22.54 

29.68 

54.18 

47.23 

$1,000 - 
$2,999 

188 

19.81 

36.61 

56.99 

58.57 

18.55 

36.12 

58.50 

57.94 

$3,000 - 
$ 9, 999 

457 

21.82 

43.39 

68.31 

68.31 

22.23 

42.11 

64.49 

65.95 

$10,000 - 
$24,999 

354 

22.65 

51.17 

76.41 

87.26 

22.80 

46.58 

73.03 

80.99 

$25,999 - 
$49,999 

155 

31.08 

52.64 

83.27 

117.85 

27.59 

52.15 

92.54 

114.79 

$50,000 
and above 

156 

28.73 

58.87 

92.58 

144.00 

26.37 

57.19 

88.90 

140.44 



Table 18 summarizes the results of the regression 
analysis. It can be observed that there is improvement 
with the inclusion of categorical variables, especially 
with FSC. From both Table 17 and Table 18, it is observed 
that Model IV yield the best predictive power with highest 
R 2 and lowest prediction error among all four models. 


Table 18. Summary of Regression Analysis 


Model 

Equation 

R 2 

I 

Y t = X ( . exp(/? 0 + A log X ( .) 

0.2347 

II 

Yi = X, exp(p 0 + /?, log X,. + A^ 5998 ./ + A F 1560 ,i 

~*~A^1615,i “*“A^'l620,/' A F] 680,/ A ^6625,1 A ^584 l,i ) 

0.2954 

III 

t= x i expfe + A l°gX, +p 2 L Ei +p 7> L u . 

+ p A L x .+p s L Y i +p 6 L A<i log X i+ j3 7 L Y . log x) 

0.2699 

IV 

Y t = X; exp (A + P x logX ; + A^ 560 ,, + Pi^E,i + Pp-V,i 
+ P 5 F 599 »,i l°gX ; + A^ 1560,1 logX, +P 7 F 1650i logx,. +P s F mo . logX, 

+ p9 F 5S95,i logX,. + Ao Laj logX,. +P n L U4 log X ( . ) 

0.3146 


However the most substantial improvement is found by 
comparing Model II to Model I. The percentage of variance 
explained in the response variable increases from 23.5 
percent to 29.5 percent with the addition of FSC levels as 
predictor variables. The practical effect of this 
improvement can be seen with the 240 items in the sample 
that belong to the FSC category "5998". For these items, 
the median (50th percentile) and upper quartile (75th 
percentile) of absolute relative errors were 42.8 percent 
and 70.5 percent for Model II, compared to 47.3 percent and 
114.6 percent respectively for Model I. 
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Figure 7 shows a scatter plot of the residuals against 
the predicted values for Model II. Ideally, this plot does 
not reveal a nonlinear pattern or evidence of unequal 
variances (heteroscedasticity). The scatter plot does 
reveal some evidence that the variance of the residuals 
decreases with the size of the predicted value. This 
pattern suggests that greater efficiency may be realized by 
using weighted least-squares regression, although ordinary 
least squares also provides unbiased estimates of the 
regression coefficients. The normal quantile-quantile (QQ) 
plot of the residuals, shown in Figure 8, indicates that 
the distribution of residuals deviates from normality 
mainly due to skewness in the left-hand tail. Despite 
these departures from ideal model assumptions. Model II 
appears to be a good fit for this analysis, and an 
improvement over the methodology currently in use. 

The next section will consider how a change in the 
prediction methodology would have affected assessment of 
the FIRST program. 
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Figure 8. Quantile-normal Plot of Residuals from Model II 



Quantiles of Standard Normal 


E. VALIDATING THE ANALYSIS WITH DATA FROM THE FIRST 

PROGRAM 

For the FIRST Program, BCA analysis was performed by 
NAVICP to estimate the repair prices of its inventory 
items. Twenty items were reviewed in which all belongs to 
the last three replacement price categories (Table 19) . It 
is observed that current NAVICP prediction methodology has 
the tendency to overestimate repair prices especially in 
this category, whereby these prediction errors had the 
greatest fiscal impact. Details of the estimated repair 
prices using NAVICP's BCA analysis are shown in Table 20. 
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DoD IG report claimed that NAVICP's BCA repair prices 
overstated the actual repair price [1]. 


Table 19. Category listings and sample size of the repair items 

reviewed by DoD IG 


Category 

Sample Size 

$10, 000 - $24, 999 

3 

$25,999 - $49,999 

5 

$50,000 and above 

12 


Table 21 shows the analysis performed using the 
current, modified and regression method as discussed in 
previous section to determine the predictive power of the 
respective models. Comparing current prediction method 
adopted by NAVICP ($65.5 million) to the proposed 
regression method ($35.8 million), the overstatement of 
benefit from the PBL that DoD IG claims would have been 
reduced by $29.7 million. This shows that the 
relationship of the repair prices to replacement prices is 
indeed non-linear. Hence using the regression method would 
reduce the prediction error of repair prices by almost $30 
million. 
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Table 20. Analysis of NAVICP BCA repair prices (FIRST Program) 


FSC 

NUN 

5 Yr 

Demand 

Repair 

Prices 

Total Cost 

1430 

014553659 

164 

$13,303 

$2,181,651 

1620 

014636970 

109 

$8,115 

$884,573 

1620 

014668717 

77 

$41,987 

$3,232,972 

1630 

014551442 

176 

$7,970 

$1,402,794 

1650 

014552590 

123 

$9,130 

$1,123,039 

1650 

014553668 

200 

$7,704 

$1,540,846 

1650 

014554490 

514 

$20,850 

$10,716,900 

1650 

014691468 

217 

$15,339 

$3,328,498 

1680 

014552537 

469 

$5,001 

$2,345,375 

1680 

014774914 

362 

$55,121 

$19,953,784 

1680 

014782049 

105 

$8,722 

$915,810 

1680 

014800498 

59 

$38,632 

$2,279,289 

1720 

014551420 

219 

$3,990 

$873,786 

2520 

014726137 

66 

$28,763 

$1,898,325 

4320 

014545082 

39 

$14,664 

$571,896 

4320 

014552588 

199 

$7,834 

$1,558,892 

5998 

012960824 

141 

$3,622 

$510,646 

5998 

014658626 

532 

$5,118 

$2,723,031 

6115 

014553692 

104 

$38,484 

$4,002,336 

6615 

014820902 

157 

$22,606 

$3,549,079 


Table 21. Comparison of Predicted Repair Prices between Current, 

Modified and Regression Method (Model II). 


Prediction Method 

Total Predicted 
Repair Price 

Total Actual 
Repair Price 

Difference 

Percent 

Prediction using NAVICP BCA 
in IGDOD Report 

$65.6 million 

$21.4 million 

$44.2 million 

206 

Prediction using Current 
Prediction Method 

$68.9 million 

$21.4 million 

$47.5 million 

222 

Prediction using Modified 
Ratio Method 

$55.8 million 

$21.4 million 

$34.4 million 

161 

Prediction using Regression 
Method (Model II) 

$35.9 million 

$21.4 million 

$14.5 million 

67 


Note: Analysis is based on the 20 items identified in 
DoD IG report [1] . 
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F. 


VALIDATING ANALYSES WITH NEWLY PROVISIONED ITEMS FOR 
FY 2002 


In addition to validating the regression methodology 
with the repair-price data from the FIRST program, an 
analysis was also conducted with newly-provisioned items 
for FY 2002. After using the filtering techniques 
discussed in Chapter III, there were a total of 575 newly 
provisioned inventory items for FY 2002 that were available 
for analysis. 


Table 22 summarizes absolute relative prediction 
errors for the current, modified and regression prediction 
(Model II) methodologies. There is no clear pattern that 
suggests that one methodology outperforms the others across 
all categories of items. It is of interest to note that, 
in the highest-cost category, the regression method 
produces the largest median of absolute relative prediction 
errors, but also the smallest 75 th percentile of these same 
quantities. 
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Table 22. Quantiles of Absolute Relative Prediction Errors (Percentages) Using the Current, 
Modified and Regression Method (Model II) for Newly Provisioned FY 2002 Inventory 

Items. 


Category 

Sample 

Size 

Current Ratio Method 

Modified Ratio Method 

Regression Method (Model II) 

25 ,n 

Percentile 

Median 

75th 

Percentile 

Mean 

25th 

Percentile 

Median 

75th 

Percentile 

Mean 

25th 

Percentile 

Median 

75th 

Percentile 

Mean 

up to 
$999 

32 

26.91 

48.34 

86.10 

64.15 

23.66 

44.54 

82.61 

61.06 

32.20 

46.58 

87.89 

98.18 

$1,000 - 
$2,999 

71 

26.60 

48.69 

79.34 

90.23 

24.86 

49.71 

79.75 

87.99 

23.07 

40.63 

76.15 

62.95 

$3,000 - 
$9,999 

173 

17.02 

35.77 

78.77 

82.39 

18.07 

37.19 

73.55 

78.14 

20.28 

42.88 

72.63 

63.03 

$10,000 

$24,999 

128 

18.88 

44.95 

73.71 

75.95 

19.63 

44.83 

72.10 

75.04 

26.50 

50.30 

77.37 

80.90 

$25,999 

$49,999 

54 

40.49 

61.81 

116.64 

104.68 

39.83 

62.75 

119.02 

105.87 

22.62 

49.09 

72.89 

68.29 

$50,000 

and 

above 

117 

6.69 

42.11 

89.90 

162.42 

17.35 

35.98 

72.06 

128.51 

22.75 

42.19 

72.77 

68.76 
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V. 


CONCLUSIONG AND RECOMMENDATIONS 


A. INTRODUCTION 


The intention of this research is to develop a 
predictive model for estimation of repair prices. This 
research was initiated after DoD IG instigated a review of 
the FIRST Program awarded to Boeing whereby the Navy 
largely underestimated the repair prices. It was observed 
that of the 20 inventory items reviewed by DoD IG, twelve 
are from the highest price category as shown in Table 19. 

B. CONCLUSION AND RECOMMENDATIONS 


In this research, three methodologies were considered. 
The first, a modified ratio methodology, is somewhat 
similar to the current ratio methodology used by NAVICP. 
The second is based on a regression with logarithmic 
transformations of both the response and predictor 
variables, and the third is an extension which includes 
categorical variables in regression analyses. 

It is observed that the largest share of the overall 
prediction error occurs with items having the largest 
replacement prices ($10,000 and above). But, in the 
highest cost category, there is noticeable improvement if 
the modified ratio methodology is used. Using the modified 
ratio methodology, the difference between predicted and 
actual repair prices was reduced to approximately $34 
million. However, this error is still large in light of 
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the fact that the actual repair price of the items 
considered was only $21 million. This finding suggests 
that there may have been much to be gained by examining 
alternative prediction methodologies. 


One suggestion highlighted in this research is the use 
of transformed regression analysis. As shown in Figure 5, 
the relationship of replacement to repair price is fairly 
well correlated on a logarithmic scale. Additional 
analysis using categorical variable are also presented in 
Chapter IV. 


Box plots for the regression methods, as shown in 
Figure 6, all indicates improvement in the higher price 
category, especially model IV (i.e. regression analysis 
with both FSC and LRC as categorical variable) which yield 
the best results. However it can be noted that all 
regression methodology though performed fairly well, had 
hardly any vast distinctive improvement among each other, 
even with the inclusion of categorical variables. Thus it 
seems that analysis with regression methodology should 
suffice for the estimation of repair prices. 


Similarly, the predictive ability of the regression 
method is tested with the repair items reviewed by DoD IG 
[1]. This model presented significant improvement in the 
predictive ability and the prediction error drops from an 
observed $47 million (using current ratio method) to about 
$14 million. 
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It is recommended that NAVICP consider the use of 
regression analysis with logarithmic transformations to 
develop it repair matrix. These models are able to 
incorporate predictors other than the replacement price, 
including FSC and LRC, which were found to be of some 
value. With a concerted research effort it is possible 
that other effective predictors may also be found. 
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APPENDIX A: LISTING OF FSC AND LRC GROUPING WITH 

DESCRIPTION 


FSC Code and Description 

1210-1290 Fire Control Equipment 
1305-1398 Ammunition and Explosives 
1410-1450 Guided Missiles 

1560-1560 Aircraft Airframe Structural Components 
1610-1615 Aircraft Propellers and Helo Rotor Blades & 
Drive Mechanisms 

1620-1680 Aircraft Components and Accessories 
1710-1990 Aircraft Launch Land & Ground Service Equip 
2010-2090 Ship & Marine Equipment 
2210-2350 Railway Equipment 
2410-2430 Tractors 

2510-2590 Vehicular Equipment Components 
2610-2640 Tires and Tubes 
2805-2895 Engines Turbines & Components 
2910-2995 Engine Accessories 

3010-3040 Mechanical Power Transmission Equipment 
3110-3130 Bearings 

3210-3230 Woodworking Machinery & Equipment 
3405-3470 Metalworking Machinery 
3510-3590 Service And Trade Equipment 
3605-3695 Special Industry Machinery 
3710-3770 Agricultural Machinery & Equipment 
3805-3895 Construction, Mining, & Excavating 
3910-3990 Materials Handling Equipment 
4010-4030 Rope, Cable, Chain, & Fittings 
4110-4140 Refrigeration & Air Conditioning 
4210-4240 Fire Fighting, Rescue & Safety Equipment 
4310-4330 Pumps and Compressors 

4410-4470 Furnace, Steam Plant & Drying Equipment 
4510-4540 Plumbing, Heating & Sanitation Equipment 
4610-4630 Water Purification/Sewage Treatment Equipment 
4710-4730 Pipe, Tubing, Hose, & Fittings 
4810-4820 Valves 

4910-4960 Maintenance & Repair Shop Equipment 
5110-5180 Hand Tools 
5210-5280 Measuring Tools 
5305-5365 Hardware and Abrasives 

5410-5450 Prefabricated Structures & Scaffolding 
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FSC Code and Description 

5510-5530 Lumber, Millwork, Plywood & Veneer 

5610-5680 Construction & Building Materials 

5805-5895 Communication & Detection Equipment 

5905-5999 Electrical & Electronic Equipment Components 

6010-6080 Fiber Optics Materials & Components 

6105-6150 Electric Wire, Power & Distribution Equipment 

6210-6260 Lighting Fixtures and Lamps 

6310-6350 Alarm, Signal & Security Detection Systems 

6505-6550 Medical, Dental Equipment & Supplies 

6605-6695 Instruments & Laboratory Equipment 

6710-6780 Photographic Equipment 

6810-6850 Chemicals & Chemical Products 

6910-6940 Training Aids & Devices 

7010-7050 Data Process & Support Equipment 

7105-7195 Furniture 

7210-7290 Household, Commercial Furnishings& Appliances 
7310-7360 Food Preparation & Serving Equipment 
7420-7490 Office Machines, Text Proc Equip & Visible 
Record Equipment 

7510-7540 Office Supplies & Devices 

7610-7690 Books, Maps & Other Publications 

7710-7740 Musical, Phonographs & Home-Type Radios 

7810-7830 Recreational & Athletic Equipment 

7910-7930 Cleaning Equipment & Supplies 

8010-8040 Brushes, Paints, Sealers & Adhesives 

8105-8145 Containers, Packaging, & Packing Supplies 

8305-8345 Textiles, Apparel, Shoe Findings-Tents&Flags 

8405-8475 Clothing, Individual Equipment & Insignia 

8510-8540 Toiletries 

8710-8730 Agricultural Supplies 

8810-8820 Live Animals 

8900-8999 Subsistence-Food 

9110-9160 Fuels, Lubricants, Oils & Waxes 

9310-9390 Nonmetallic Fabricated Materials 

9410-9450 Nonmetallic Crude Materials 


9505-9545 Metal Bars, Sheets, and Shapes 
9610-9680 Ores, Minerals & Their Primary Products 
9905-9999 Miscellaneous 
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LRC Code and Platform 

A F/A-18 A/D 

B F/A-18 E/F 

C AV-8B 
D EA-6B 
E F-14 
G V-22 
H S-3 

J C-130,E-6,F-5,F-16,T-38 
L P-3 
M E-2 
P C-2 

Q Common Systems 
R T-45 

S Aircraft Engines 

T/U Aviation Support Systems 

V H-l/H-46 
X H-3/H-53 

Y H-60 
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